The stability-plasticity dilemma: investigating the continuum from catastrophic forgetting to age-limited learning effects
نویسندگان
چکیده
The stability-plasticity dilemma is a wellknow constraint for artificial and biological neural systems. The basic idea is that learning in a parallel and distributed system requires plasticity for the integration of new knowledge, but also stability in order to prevent the forgetting of previous knowledge. Too much plasticity will result in previously encoded data being constantly forgotten, whereas too much stability will impede the efficient coding of this data at the level of the synapses. However, for the most part, neural computation has addressed the problems related to excessive plasticity or excessive stability as two different fields in the literature.
منابع مشابه
Alleviating Catastrophic Forgetting via Multi-Objective Learning [IJCNN1762]
Handling catastrophic forgetting is an interesting and challenging topic in modeling the memory mechanisms of the human brain using machine learning models. From a more general point of view, catastrophic forgetting reflects the stability-plasticity dilemma, which is one of the several dilemmas to be addressed in learning systems: to retain the stored memory while learning new information. Diff...
متن کاملMethods for reducing interference in the Complementary Learning Systems model: Oscillating inhibition and autonomous memory rehearsal
The stability-plasticity problem (i.e. how the brain incorporates new information into its model of the world, while at the same time preserving existing knowledge) has been at the forefront of computational memory research for several decades. In this paper, we critically evaluate how well the Complementary Learning Systems theory of hippocampo-cortical interactions addresses the stability-pla...
متن کاملPii: S0893-6080(01)00018-1
As an extension of on-line learning, life-long learning challenges a system which is exposed to patterns from a changing environment during its entire lifespan. An autonomous system should not only integrate new knowledge on-line into its memory, but also preserve the knowledge learned by previous interactions. Thus, life-long learning implies the fundamental Stability±Plasticity Dilemma, which...
متن کاملPseudo-recurrent Connectionist Networks: An Approach to the 'Sensitivity-Stability' Dilemma
In order to solve the “sensitivity-stability” problem — and its immediate correlate, the problem of sequential learning — it is crucial to develop connectionist architectures that are simultaneously sensitive to, but not excessively disrupted by, new input. French (1992) suggested that to alleviate a particularly severe form of this disruption, catastrophic forgetting, it was necessary for netw...
متن کاملThe Neuronal Replicator Hypothesis
We propose that replication (with mutation) of patterns of neuronal activity can occur within the brain using known neurophysiological processes. Thereby evolutionary algorithms implemented by neuro- nal circuits can play a role in cognition. Replication of structured neuronal representations is assumed in several cognitive architectures. Replicators overcome some limitations of selectionist mo...
متن کامل